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The National Center for Education Statistics (NCES) fulfills a congressional 
mandate to collect and report “statistics and information showing the con- 
dition and progress of education in the United States and other nations in 
order to promote and accelerate the improvement of American education.” 



Education Statistics Quarterly 

Purpose and goals 

At NCES, we are convinced that good data lead to good decisions about 
education. The Education Statistics Quarterly is part of an overall effort to 
make reliable data more accessible. Goals include providing a quick way to 

■ identify information of interest; 

■ review key facts, figures, and summary information; and 

■ obtain references to detailed data and analyses. 

Content 

The Quarterly gives a comprehensive overview of work done across all 
parts of NCES. Each issue includes short publications, summaries, and 
descriptions that cover all NCES publications and data products released 
during a 3-month period. To further stimulate ideas and discussion, each 
issue also incorporates 

■ a message from NCES on an important and timely subject in 
education statistics; and 

■ a featured topic of enduring importance with invited commentary. 

A complete annual index of NCES publications appears in the fourth issue of 
each volume. Publications in the Quarterly have been technically reviewed for 
content and statistical accuracy. 



General note about the data and interpretations 



Many NCES publications present data that are based 
on representative samples and thus are subject to 
sampling variability. In these cases, tests for statistical 
significance take both the study design and the number 
of comparisons into account. NCES publications only 
discuss differences that are significant at the 95 percent 
confidence level or higher. Because of variations in 
study design, differences of roughly the same magnitude 
can be statistically significant in some cases but not in 
others. In addition, results from surveys are subject to 



nonsampling errors. In the design, conduct, and 
data processing of NCES surveys, efforts are made to 
minimize the effects of nonsampling errors, such as 
item nonresponse, measurement error, data processing 
error, and other systematic error. 

For complete technical details about data and meth- 
odology, including sample sizes, response rates, and 
other indicators of survey quality, we encourage readers 
to examine the detailed reports referenced in each article. 
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Note From NCES 

Peggy G. Carr, Associate Commissioner, Assessment Division 



Evaluating the Performance of Urban Students 

One of the leading issues in American education today is how well students in the nation’s 
large urban areas are performing. Many large urban school districts are located in states in 
which they are the only such district. When their students are tested using a state test, 
these districts can be compared with the rest of the state’s population, but not with large 
urban districts in other states. The National Assessment of Educational Progress (NAEP) is 
the only assessment system that can provide comparable statistics across state boundaries. 
To examine the feasibility and value of using NAEP to compare large urban districts in 
various states, the National Center for Education Statistics (NCES) inaugurated the Trial 
Urban District Assessment (TUDA). TUDA provides results for fourth- and eighth-graders 
in selected urban districts using the same NAEP assessment questions administered at the 
national and state levels. In 2002, TUDA was conducted for the first time, with assess- 
ments in reading and writing. Results from 2002 are featured in this issue of the Quarterly. 
In 2003, TUDA was conducted again, with assessments in reading and mathematics. 



Characteristics of TUDA Districts 

In 2002, NCES invited five large urban districts to participate in TUDA: New York City 
Public Schools, Los Angeles Unified, Chicago School District 299, Houston Independent 
School District, and Atlanta City. In 2003, four more districts were added: Boston School 
District, Charlotte-Mecklenburg Schools (in North Carolina), Cleveland Municipal School 
District, and San Diego City Schools. TUDA required NCES to work closely with school 
district administrators, as well as with school principals and staff. I appreciate the help that 
the Council of the Great City Schools provided in working with these large school dis- 
tricts. (In addition to results for the nine selected districts, results for the District of 
Columbia, which has participated in many NAEP state assessments, are included in TUDA 
reports.) 



The nine districts were chosen to provide variation in characteristics such as student 
population size, demographic and socioeconomic composition, and geographic location. 
The size of these districts ranged from about 55,000 students in Atlanta to over one 
million in New York City. Data from the 2003 TUDA in mathematics at grade 4 show the 
largest proportion of Black students in Atlanta (87 percent), the largest proportion of 
Hispanic students in Los Angeles (73 percent), and the largest proportion of White 
students in Charlotte-Mecklenburg (41 percent). The proportion of limited-English- 
proficient (LEP) fourth-graders ranged from 2 percent in Atlanta to 56 percent in Los 
Angeles. All of the fourth-graders in Cleveland were eligible for free or reduced-price 
school lunch — an indicator of poverty status — while 45 percent of those in Charlotte- 
Mecklenburg were eligible. The large urban districts were distributed across all four 
regions of the United States. While most of the districts were located entirely within large 
cities, two districts, Charlotte-Mecklenburg and Los Angeles Unified, also included schools 
located in suburban areas. 
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Not only did the large urban districts differ from each other, but they also tended to differ 
from the nation as a whole and from the states. For example, all of the participating 
districts had lower proportions of White students than did the nation. And all of the 
districts except Charlotte-Mecklenburg had considerably higher proportions of students 
eligible for free or reduced-price school lunch than did the nation. The dramatic differ- 
ences between the characteristics of a given urban school district and those of the nation, 
the states, and other districts create special challenges in analyzing and reporting the data. 



Challenges in Analyzing and Reporting TUDA Data 

Because of differences in characteristics, it can be difficult to make meaningful compari- 
sons between TUDA districts and the nation or the states. To provide a more appropriate 
frame of reference, NCES also compared the 2002 results for TUDA districts with the 
average for all students in central cities across the nation. The term “central city” does not 
mean the central part of a city or the “inner city,” but rather a city that is central. A central 
city is defined by the Census Bureau as a city of 50,000 people or more that is the largest in 
its metropolitan area, or can otherwise be regarded as central, taking into account such 
characteristics as commuting patterns. However, many central cities — for example, 

Lawton, Oklahoma, or Parkersburg, West Virginia — do not resemble the TUDA districts to 
any great degree. In 2003, therefore, NCES refined its comparison group for the TUDA 
districts to include only large central cities (defined as central cities with populations of at 
least 250,000). About 15 percent of the nation’s students live in large central cities. In 
terms of student population characteristics, TUDA districts are often more similar to the 
large central cities than to the nation or the states. 



Another way in which NCES addresses the issue of widely varying characteristics is by 
comparing the performance of individual subgroups in a given TUDA district with the 
performance of the same subgroups elsewhere. For example, the performance of Black 
fourth-graders in Atlanta can be compared with that of Black fourth-graders in the nation, 
in the large central cities, in Georgia, and in other TUDA districts. Similarly, the perfor- 
mance of students eligible for free or reduced-price lunch can be compared with the 
performance of other eligible students. Such comparisons frequently demonstrate that 
subgroup members in TUDA districts are performing at or above the national average for 
their peers (though of course subgroup members in some TUDA districts are performing 
below this average). 



An additional issue concerns differences in the proportions of students that TUDA districts 
excluded from the assessments, either because those students had disabilities that pre- 
vented their participation, or because their knowledge of English was not sufficient for 
them to participate meaningfully. In the 2003 TUDA assessment in reading, for example, 

33 percent of Houston’s fourth-grade sample was identified as LEP and 20 percent of the 
sample was excluded. (Houston typically assesses the reading skills of LEP students in 



EDUCATION STATISTICS QUARTERLY — VOLUME 5, ISSUE 3, 2003 



5 



Spanish, while NAEP assesses reading skills only in English.) In Los Angeles, 56 percent of the sample 
was identified as LEP, but only 5 percent of the sample was excluded. (In California, state law requires 
testing of nearly all students.) Variability in the proportion of students excluded should be taken into 
consideration when interpreting the results and making comparisons. 



Available Results and Future Plans for TUDA 

One of the primary values of NAEP to the public is the ability to compare performance in various 
jurisdictions with performance in other jurisdictions and in the nation. While NCES has always 
administered NAEP assessments to students in urban areas, TUDA for the first time allows comparisons 
of students in individual large urban districts to students in the nation, in large central cities (taken as a 
whole), in the states, and in other participating districts. Such comparative data appear in NCES reports 
on the 2002 and 2003 TUDA, and one can also make one’s own comparisons using the NAEP Data Tool 
available on the NCES web site ( http://nces.ed.gov/nationsreportcard/naepdata ). 



I am excited about the opportunity to provide the public with comparative information about student 
performance in large urban districts. Additional information will be available after TUDA is conducted 
again in 2005. However, the program retains its trial status. The decision whether to make urban 
district assessments a permanent part of NAEP will be made only after NCES and the National Assess- 
ment Governing Board (NAGB) further evaluate the results. 
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The Nation’s Report Card: Reading 2002, Trial Urban District Assessment 

Anthony D. Lutkus, Arlene W Weiner, Mary C. Daane, and Yingjin 

This article was originally published as the Executive Summary of the Statistical Analysis Report of the same name. The sample survey data are from the 
National Assessment of Educational Progress (NAEP) 2002 Trial Urban District Reading Assessment and 2002 Reading Assessment. 



The National Assessment of Educational Progress (NAEP) is 
the nation’s ongoing representative sample survey of student 
achievement in core subject areas. NAEP, known as the 
Nation’s Report Card, is authorized by Congress and 
administered by the National Center for Education Statistics 
(NCES) of the Institute of Education Sciences in the U.S. 
Department of Education. NAEP regularly reports to the 
public on the educational progress of students in grades 
4, 8, and 12. 

In 2002, NAEP assessed the reading and writing perfor- 
mance of the nation’s fourth-, eighth-, and twelfth-grade 
students. NAEP also conducted assessments of fourth- and 
eighth-graders’ reading and writing in most of the states. 



In 2001, after discussion among NCES, the National 
Assessment Governing Board (NAGB), and the leadership 
of the Council of the Great City Schools, Congress appro- 
priated funds for a trial district-level assessment and NAGB 
passed a resolution approving the selection of five large 
urban districts for participation in the Trial Urban District 
Assessment, a special project within NAEP. Thus, this report 
presents, for the first time, district-level results of NAEP 
reading assessments in five urban public school districts: 
Atlanta City, Chicago School District 299, Houston Inde- 
pendent School District, Los Angeles Unified, and New York 
City Public Schools. Throughout this report, the districts 
are referred to simply as Atlanta, Chicago, Houston, Los 
Angeles, and New York City. The five districts participated 
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voluntarily in the NAEP 2002 Trial Urban District Assess- 
ment in reading at grades 4 and 8. Results are also included 
in this report for the District of Columbia, which, in 2002 
and past years, has been included in the main assessments 
with states and other jurisdictions. Data for public schools 
across the nation and for central city public schools are 
provided for comparison purposes.* The public schools 
also included charter schools, which in some cases were not 
managed by the urban school district. 

NAEP does not provide scores for individual students or 
schools. It reports results for groups of students (e.g., 
fourth-graders). For each group in each table in the report, 
assessment results are described in one of two ways: the 
group’s average reading score on a scale from 0 to 500 or the 
percentage of students in the group who reached each of 
three achievement levels: Basic, Proficient, and Advanced. 

The Proficient level for each grade is defined by NAGB as 
representing “solid academic performance,” which demon- 
strates “competency over challenging subject matter” for 
the grade assessed. Basic indicates partial mastery of skills 
that are fundamental for proficient work. Advanced denotes 
superior performance. 

The achievement levels are performance standards adopted 
by NAGB as part of its statutory responsibilities. The 
achievement levels are a collective judgment of what 
students should know and be able to do for each grade 
tested. As provided by law, NCES, upon review of a congres- 
sionally mandated evaluation of NAEP, determined that the 
achievement levels are to be used on a trial basis and should 
be interpreted with caution. However, both NCES and 
NAGB believe that the performance standards are useful for 
understanding trends in student achievement. They have 
been widely used by national and state officials and others 
as a common yardstick of academic performance. 

The results are based on representative samples of students 
for the nation, for schools in central cities, and for partici- 
pating districts. In order to obtain reliable and representa- 
tive data, a large proportion of the selected schools and 
students must participate. All six districts met the NCES 
statistical participation criteria for NAEP samples at grade 
4, but New York City data are not reported for grade 8 
because eighth-grade participation did not meet the criteria. 



*"Central city" includes a nationally representative sample of public schools located in 
central cities within metropolitan statistical areas as defined by the federal Office of 
Management and Budget. A central city is a city of 50,000 or more that is the largest in 
its metropolitan area, or can otherwise be regarded as"central."The term means "a city 
that is central," not "the central part of a city" or the "inner city." Note that central cities 
encompass wider areas than what is commonly referred to as "the inner city." 



Some students are identified by the school districts as 
students with disabilities or limited-English-proficient 
students. Some of these students are excluded from the 
assessment, and others are tested with accommodations 
related to their status. Because the percentages of students 
identified, excluded, and assessed with accommodations 
vary across the urban districts, that variability should be 
taken into consideration in interpreting the results and 
making comparisons. 

For example, in the case of fourth-grade students, the 
percentages of students identified as having disabilities or 
limited English proficiency ranged from 8 to 51 percent, the 
percentages of fourth-grade students excluded for these 
reasons ranged from 2 to 17 percent, and the percentages 
assessed with accommodations ranged from 1 to 8 percent. 
At the eighth grade, the percentages of students identified 
with disabilities or limited English proficiency ranged from 
6 to 35 percent, the percentages of eighth-grade students 
excluded for these reasons ranged from 2 to 7 percent, and 
the percentages assessed with accommodations ranged from 
0 to 8 percent. 

Overall Reading Results for the Urban Districts 

Average scale scores 
Results for grade 4 

■ The average scale scores for fourth-graders ranged 
from 191 in the District of Columbia and Los Angeles 
to 206 in Houston and New York City. 

■ The average score for public school students in the 
nation as a whole was higher than the average score 
in each of the urban districts, and the national 
average score in central city public schools was 
higher than the average score in each of the urban 
districts except Houston and New York City (figure A). 

■ The average scale scores in Houston and New York 
City were higher than those of the other urban 
districts and were not found to differ significantly 
from each other. 

Results for grade 8 

Results for New York City schools at grade 8 are not 
reported because they did not meet participation rates. 

■ The average scale scores for eighth-graders ranged 
from 236 in Atlanta to 249 in Chicago. 

■ The national average scores for public school stu- 
dents and for students in the central city public 
schools were higher than the average score in any of 
the urban districts (figure B). 
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The Nation's Report Card: Reading 2002, Trial Urban District Assessment 



Figure A. Cross-district comparisons of average reading scale scores, grade 4 public schools: By urban 
district, 2002 



Instructions: Read across the row corresponding to a district listed to the left of the chart. Use the key 
to determine whether the average reading scale score of this district was found to be higher than, not 
significantly different from, or lower than that of the jurisdiction in the column heading. For example, 
in the row for Atlanta: Atlanta's score was lower than the scores of the nation, the central city sample, 
New York City, and Houston; not significantly different from the scores of Chicago and Los Angeles; 
and higher than the score of the District of Columbia. 




m 



fl 



District had higher average 
scale score than the jurisdiction 
listed at the top of the figure. 

No significant difference 
detected from the jurisdiction 
listed at the top of the figure. 

District had lower average scale 
score than the jurisdiction listed 
at the top of the figure. 



^Although deemed sufficient for reporting, the target response rate specified in the NAEP guidelines was not met. 

NOTE:The between-district comparisons take into account sampling and measurement error and that each district is 
being compared with every other district shown. Significance is determined by an application of a multiple-comparison 
procedure. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational 
Progress (NAEP) 2002 Trial Urban District Reading Assessment and 2002 Reading Assessment. (Originally published as 
figure 2.1 on p. 1 4 of the complete report from which this article is excerpted.) 



Figure B. Cross-district comparisons of average reading scale scores, grade 8 public schools: By urban 
district, 2002 



Instructions: Read across the row corresponding to a district listed to the left of the chart. Use the 
key to determine whether the average reading scale score of this district was found to be higher 
than, not significantly different from, or lower than that of the jurisdiction in the column heading. 
For example, in the row for the District of Columbia: the District of Columbia's score was lower than 
the scores of the nation, the central city sample, Chicago, and Houston; it was higher than the 
scores of Los Angeles and Atlanta. 



Chicago 
Houston 
District of Columbia 
Los Angeles 
Atlanta 




III a | District had higher average 
— scale score than the jurisdiction 
listed at the top of the figure. 




No significant difference 
detected from the jurisdiction 
listed at the top of the figure. 

District had lower average scale 
score than the jurisdiction listed 
at the top of the figure. 



NOTE:The between-district comparisons take into account sampling and measurement error and that each district is 
being compared with every other district shown. Significance is determined by an application of a multiple-comparison 
procedure. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational 
Progress (NAEP) 2002 Trial Urban District Reading Assessment and 2002 Reading Assessment. (Originally published as 
figure 2.2 on p. 1 5 of the complete report from which this article is excerpted.) 
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■ The average scale scores in Chicago and Houston 
were higher than those of the other urban districts 
and were not found to differ significantly from each 
other. 

Reading achievement levels 

Results for grade 4 

■ The percentages of fourth-graders performing at or 
above Proficient ranged from 10 percent in the 
District of Columbia to 19 percent in New York City. 
The percentage of students performing at or above 
Proficient in public schools in the nation was 30 per- 
cent, and for students in central city public schools it 
was 21 percent. 

■ Any apparent differences between the percentages of 
students performing at or above Proficient in the 
urban districts were not found to be statistically 
significant. 

■ The percentages of students performing at or above 
Basic ranged from 31 percent in the District of 
Columbia to 48 percent in Houston. In public 
schools across the nation, 62 percent of students 
performed at or above Basic. In central city schools 
in the national sample, 51 percent performed at or 
above Basic. 

■ The percentages of students performing at or above 
Basic were higher in Houston and New York City 
than in the other urban districts. 

Results for grade 8 

■ The percentages of eighth-graders performing at or 
above Proficient ranged from 8 percent in Atlanta to 
17 percent in Houston. Thirty-one percent of stu- 
dents in public schools in the nation and 23 percent 
in central city public schools performed at or above 
Proficient. 

■ The percentages of students performing at or above 
Proficient in national public schools and central city 
public schools were higher than the percentages in 
each of the urban districts. 

■ The percentages of students performing at or above 
Basic ranged from 42 percent in Atlanta to 62 percent 
in Chicago. Seventy-four percent of public school 
students in the nation and 64 percent in central cities 
performed at or above Basic. 

■ The percentages of students performing at or above 
Basic in Chicago and Houston did not differ signifi- 



cantly from each other and both were higher than the 
comparable percentages in Atlanta, the District of 
Columbia, and Los Angeles. 

Results for Student Subgroups 

In addition to providing average scores and achievement 
levels for the nation, for states, and, in this report, for urban 
districts, NAEP reports provide results for subgroups of 
students defined by various background and contextual 
characteristics (e.g., gender, eligibility for free or reduced- 
price lunch, and level of parents’ education). 

Gender 

Results for grade 4 

■ Both male and female fourth-grade students in 
Atlanta, Chicago, the District of Columbia, and Los 
Angeles had average scores that were below the 
national average score for their counterparts in 
central city public schools. 

■ Female students in the urban districts outscored male 
students, on average. The exception was Houston, 
where the apparent difference was not statistically 
significant. 

Results for grade 8 

■ The average score for male eighth-graders in central 
city public schools across the nation was higher than 
the average for male eighth-graders in each of the 
urban districts. The average score for female students 
in all urban districts except Chicago was below the 
national average for female students in central city 
public schools. 

■ In all urban districts, female students had higher 
average scores than male students. 

Race/ethnicity 

Whereas White students constituted 60 percent of the 
national public sample at grade 4 and 64 percent at grade 8, 
in the urban districts, White students made up a maximum 
of 15 percent of the samples at grade 4 (New York City) and 
11 percent of the samples at grade 8 (Chicago). Black or 
Hispanic students constituted majorities in the urban 
districts in the trial assessment. Hispanic students made up 
half or more of the sample in Houston and Los Angeles at 
both grades 4 and 8. Black fourth- and eighth-grade 
students made up more than 80 percent of the sample in 
both Atlanta and the District of Columbia. 
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Results for grade 4 

■ In the five urban districts in which a reliable com- 
parison could be made, White fourth-graders had 
higher average scores than their Black and Hispanic 
peers. In Chicago, Hispanic students had higher 
average scores than Black students. 

■ The average scores for Black students in Chicago, the 
District of Columbia, and Los Angeles were lower 
than the national average for Black students in 
central city public schools. 

■ The average scores for Hispanic students in Chicago, 
Houston, and Los Angeles were lower than the 
national average for Hispanic students in central city 
public schools. 

■ The average score for Asian/Pacific Islander students 
in New York City was higher than the national 
average for Asian/Pacific Islander students in central 
city public schools. 

Results for grade 8 

■ White eighth-graders had higher average scores than 
Black eighth-graders in Atlanta, Houston, and Los 
Angeles. The apparent difference in Chicago was not 
found to be statistically significant, and the sample 
size in the District of Columbia was insufficient to 
permit a reliable comparison. 

■ The average scores for Black students in Atlanta, the 
District of Columbia, and Los Angeles were lower 
than the national average for Black students in 
central city public schools. 

■ The average score for Hispanic students in Los 
Angeles was lower than the national average for 
Hispanic students in central city public schools. 

■ The average score for White students in Houston was 
higher than the national average for White students 
in central city public schools. 

Eligibility for free/reduced-price lunch 

The federal program providing free/reduced-price school 
lunch is administered by the U.S. Department of Agriculture 
(USDA) for children near or below the poverty line. 
Eligibility is determined by the USD/Ys Income Eligibility 
Guidelines ( http :// www. fns . usda . go v/ cnd/go vernance/no tices/ 
IEGs/IEGs.htm L At grade 4, the percentages of students in 
the urban districts eligible for free/reduced-price lunch 
ranged from 72 percent in Houston to 88 percent in 
Chicago. By comparison, 43 percent of fourth-graders in 
public schools nationally were eligible. At grade 8, the percent- 



ages of students eligible for free/reduced-price lunch in four 
of the urban districts ranged from 68 percent to 84 percent. 
By comparison, 34 percent of eighth-graders in public schools 
nationally were eligible. (Information on the free/reduced- 
price lunch data for eighth-graders in Los Angeles is not 
reported because these data did not meet reporting standards.) 

Results for grade 4 

■ In each of the urban districts, fourth-grade students 
not eligible for free/reduced-price lunch had higher 
average scores than students who were eligible. 

■ The average scores for eligible students in Atlanta, 
Chicago, the District of Columbia, and Los Angeles 
were lower than the national average for eligible 
students in central city public schools. 

■ The average scores for students in Atlanta, the 
District of Columbia, and Los Angeles who were not 
eligible for the program were lower than the national 
average for students in central city public schools 
who were not eligible. 

Results for grade 8 

■ Eighth-grade students not eligible for free/reduced- 
price lunch had higher average scores than eligible 
students in each of the urban districts except Chi- 
cago, where the apparent difference was not statisti- 
cally significant. 

■ The average scores for eligible students in Atlanta 
and the District of Columbia were lower than the 
national average for eligible students in central city 
public schools. 

■ The average scores for students in Atlanta, the 
District of Columbia, and Houston who were not 
eligible for the program were lower than the national 
average for students in central city public schools 
who were not eligible. 

Parents' highest level of education 

Eighth-grade students who participated in the Trial Urban 
District Assessment were asked to indicate the highest level 
of education they thought that their parents had completed. 
Five response options were offered: did not finish high 
school, graduated from high school, some education after 
high school, graduated from college, and “I don’t know.” 

■ In comparison with the other urban districts, the 
District of Columbia had the highest percentage of 
eighth-graders (40 percent) who reported that at least 
one parent had graduated from college. 
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In each of the urban districts, the percentage of 
students who reported that at least one parent had 
graduated from college was lower than that of public 
schools nationally. 

For students who reported that at least one parent 
had graduated from college, the average scores for 
students in Atlanta, Chicago, the District of Colum- 
bia, and Los Angeles were lower than the national 
average for students in central city public schools. 



Data source: The National Assessment of Educational Progress (NAEP) 
2002 Trial Urban District Reading Assessment and 2002 Reading 
Assessment. 

For technical information, see the complete report: 

Lutkus, A.D., Weiner, A. W., Daane, M.C.,and Jin,Y. (2003). The Nation's 
Report Card: Reading 2002, Trial Urban District Assessment 
(NCES 2003-523). 

Author affiliations: A.D. Lutkus, A.W. Weiner, M.C. Daane, and Y. Jin, 
Educational Testing Service. 

For questions about content, contact Arnold Goldstein 
( arnold.aoldstein@ed.aov) . 

To obtain the complete report (NCES 2003-523), call the toll-free 
ED Pubs number (877-433-7827) or visit the NCES Electronic Catalog 
(h ttp://nces.ed.gov/pubsearch ). 
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The Nation’s Report Card: Writing 2002, Trial Urban District Assessment 

Anthony D. Lutkus, Mary C. Daane, Arlene W. Weiner, and Yingjin 

This article was originally published as the Executive Summary of the Statistical Analysis Report of the same name. The sample survey data are from the 
National Assessment of Educational Progress (NAEP) 2002 Trial Urban District Writing Assessment and 2002 Writing Assessment. 



The National Assessment of Educational Progress (NAEP) is 
the nation’s ongoing representative sample survey of student 
achievement in core subject areas. NAEP, known as the 
Nation’s Report Card, is authorized by Congress and 
administered by the National Center for Education Statistics 
(NCES) of the Institute of Education Sciences in the U.S. 
Department of Education. NAEP regularly reports to the 
public on the educational progress of students in grades 4, 

8, and 12. 

In 2002, NAEP assessed the reading and writing perfor- 
mance of the nation’s fourth-, eighth-, and twelfth-grade 
students. NAEP also conducted assessments of fourth- and 
eighth-graders’ reading and writing in most of the states. 

In 2001, after discussion among NCES, the National 
Assessment Governing Board (NAGB), and the leadership 
of the Council of the Great City Schools, Congress appro- 
priated funds for a trial district-level assessment and NAGB 
passed a resolution approving the selection of five large 
urban districts for participation in the Trial Urban District 
Assessment, a special project within NAEP. This report 
presents, for the first time, results of NAEP’s Trial Urban 
District Assessment in writing for public school students in 
the following participating urban school districts: Atlanta 
City, Chicago School District 299, Houston Independent 
School District, Los Angeles Unified, and New York City 
Public Schools. This represents NAEP’s first assessment of 
urban districts based on samples specially designed to allow 
reporting of subgroup data. The five districts participated 
voluntarily in the NAEP 2002 Trial Urban District Assess- 
ment in writing at grades 4 and 8. Results for the District of 
Columbia, which in this and past NAEP assessments has 
been sampled and assessed along with states and other 
jurisdictions, are also included in this report. Data for 
public schools across the nation and for central city public 
schools are provided for comparison purposes.* The public 
schools sampled also included charter schools, which in 
some cases were not managed by the urban school districts. 



*"Central city" includes a nationally representative sample of public schools located in 
central cities within metropolitan statistical areas as defined by the federal Office of 
Management and Budget. A central city is a city of 50,000 or more that is the largest in 
its metropolitan area, or can otherwise be regarded as"central."The term means "a city 
that is central," not "the central part of a city" or the "inner city." Note that central cities 
encompass wider areas than what is commonly referred to as "the inner city." 



NAEP does not provide scores for individual students or 
schools. It reports results for groups of students (e.g., 
fourth-graders). For each group in each table in the report, 
assessment results are described in one of two ways. First, 
the group’s average writing score is reported on a scale from 
0 to 300. Performance for each grade is scaled separately; 
therefore, average scale scores cannot be compared across 
grades. The term “average score” is used throughout this 
report to refer to the average scale score on the NAEP 
writing scale. Second, student writing performance is 
reported in terms of the percentage of students in the group 
who reached each of three achievement levels: Basic, 
Proficient, and Advanced. The Proficient level for each grade 
is defined by NAGB as representing “solid academic 
performance,” which demonstrates “competency over 
challenging subject matter” for the grade assessed. Basic 
indicates partial mastery of skills that are fundamental for 
proficient work. Advanced denotes superior performance. 

The achievement levels are performance standards adopted 
by NAGB as part of its statutory responsibilities. The achieve- 
ment levels are a collective judgment of what students should 
know and be able to do for each grade tested. As provided 
by law, NCES, upon review of a congressionally mandated 
evaluation of NAEP, determined that the achievement levels 
are to be used on a trial basis and should be interpreted 
with caution. However, both NCES and NAGB believe that 
the performance standards are useful for understanding 
trends in student achievement. They have been widely used 
by national and state officials and others as a common 
yardstick of academic performance. 

The results are based on representative samples of students 
for the nation, for participating districts, and for schools in 
central cities. In order to obtain reliable data, sufficient 
numbers of the selected schools and students must partici- 
pate in the assessment. All six districts met the NCES 
participation criteria for NAEP samples at grade 4, but 
results for New York City schools at grade 8 are not re- 
ported because they did not meet the participation criteria. 

Some students are identified by the school districts as 
students with disabilities or limited-English-proficient 
students. Some of these students are excluded from the 
assessment, and others are tested with accommodations 
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related to their status. Three of the six districts identified 
between 30 and 52 percent of their students as either 
students with disabilities or limited-English-proficient 
students. Because the percentages of students identified, 
excluded, and assessed with accommodations vary across 
the districts, that variability should be taken into consider- 
ation in interpreting the results and making comparisons. 

Overall Writing Results for the Urban Districts 

The following summary describes results first in terms of 
average scale scores and then in terms of achievement 
levels. Average results for public schools in the districts 
participating in the Trial Urban District Assessment are 
compared, at grades 4 and 8, with public schools in the 
nation, with public schools in central cities, and with each 
other. 



Average scale scores 
Results for grade 4 

■ The average scores for fourth-graders in public 
schools ranged from 135 in the District of Columbia 
to 153 in New York City and the nation. 

■ At grade 4, no statistically significant differences were 
detected between the average scores for students in 
Houston and New York City and the average score 
for students in public schools in the nation, while 
students in Atlanta, Chicago, the District of Colum- 
bia, and Los Angeles had average scores lower than 
the average score in the nation (figure A). 

■ At grade 4, the average score for students in New 
York City was higher than the national average score 
for students in central city public schools. The 
average score for fourth-graders in Houston was not 



Figure A. Cross-district comparisons of average writing scale scores, grade 4 public schools: By urban 
district, 2002 



Instructions: Read across the row corresponding to an urban district listed to the left of the chart. 

Use the key to determine whether the average writing scale score of this district was found to be higher 
than, not significantly different from, or lower than that of the jurisdiction named in the column heading. 
For example, in the row for Atlanta: Atlanta's average score was lower than the average scores in the 
nation, the central city sample, New York City, and Houston, but not found to differ significantly from the 
average scores in Los Angeles, Chicago, and the District of Columbia. 



New York City* 
Houston 
Los Angeles 
Atlanta 
Chicago 
District of Columbia 



^Although deemed sufficient for reporting, the target response rate specified in the NAEP guidelines was not met. 

NOTE:The between-district comparisons take into account sampling and measurement error and that each district is being 
compared with every district shown. Significance is determined by an application of a multiple-comparison procedure. 
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District had higher average 
scale score than the jurisdiction 
listed at the top of the figure. 



No significant difference 
detected from the jurisdiction 
listed at the top of the figure. 

District had lower average scale 
score than the jurisdiction listed 
at the top of the figure. 



SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational 
Progress (NAEP) 2002 Trial Urban District Writing Assessment and 2002 Writing Assessment. (Originally published as figure 
2.1 on p. 1 9 of the complete report from which this article is excerpted.) 
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found to differ significantly from that for central 
cities, while the average score for students in each of 
the other districts was lower than the national 
average score for central cities. 

■ The average score for students in New York City was 
higher than those in all the other participating 
districts except Houston. The average score in 
Houston was higher than the scores in Atlanta, 
Chicago, and the District of Columbia, but was not 
found to differ significantly from the average scores 
in Los Angeles and New York City. 

Results for grade 8 

Results for New York City schools at grade 8 are not 
reported because they did not meet participation criteria. 

■ The average district scores for eighth-graders ranged 
from 128 in the District of Columbia and Los Angeles 
to 138 in Houston. 



■ In each of the reported districts, the average score of 
eighth-grade students was lower than the average 
score for eighth-grade students in public schools in 
the nation (figure B). 

■ At grade 8, no significant difference was detected 
between the average score for students in Houston 
and the average score for students in the central city 
public schools. The average score in the central city 
schools was higher than the average scores in Atlanta, 
Chicago, the District of Columbia, and Los Angeles. 

■ The average score for students in Houston was higher 
than the average scores in Atlanta, the District of 
Columbia, and Los Angeles. The average score in 
Chicago was not found to differ significantly from 
those in Houston and Atlanta, and was higher than 
the average scores in the District of Columbia and 
Los Angeles. 



Figure B. Cross-district comparisons of average writing scale scores, grade 8 public schools: By urban 
district, 2002 



Instructions: Read across the row corresponding to an urban district listed to the left of the chart. 

Use the key to determine whether the average writing scale score of this district was found to be 
higher than, not significantly different from, or lower than that of the jurisdiction in the column heading. 
For example, in the row for the District of Columbia: the average score for the District of Columbia was 
lower than the average scores for the nation, the central city sample, Houston, and Chicago, but not 
found to differ significantly from the scores for Atlanta and Los Angeles. 
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Houston 
Chicago 
Atlanta 
District of Columbia 
Los Angeles 




|~4~| District had higher average 
Ijjjj scale score than the jurisdiction 
listed at the top of the figure. 

I I No significant difference 
— detected from the jurisdiction 
listed at the top of the figure. 

n District had lower average scale 
score than the jurisdiction listed 
at the top of the figure. 



NOTE:Thc between-district comparisons take into account sampling and measurement error and that each district is 
being compared with every other district shown. Significance is determined by an application of a multiple-comparison 
procedure. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational 
Progress (NAEP) 2002 Trial Urban District Writing Assessment and 2002 Writing Assessment. (Originally published as 
figure 2.2 on p. 20 of the complete report from which this article is excerpted.) 
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Writing achievement levels 
Results for grade 4 

■ At grade 4, the percentages of students performing at 
or above Proficient ranged from 11 percent in the 
District of Columbia to 27 percent in New York City. 

■ The percentages of fourth-grade students performing 
at or above Proficient in Houston and New York City 
were not found to be significantly different from the 
percentages in public schools in the nation or in 
central cities. Atlanta, Chicago, the District of 
Columbia, and Los Angeles had lower percentages of 
students at or above Proficient than the nation and 
central cities. 

■ At grade 4, the percentage of students performing at 
or above Proficient in New York City was higher than 
the percentages in four other districts and not found 
to differ significantly from the percentage in Houston. 

■ The percentages of fourth-grade students performing 
at or above Basic ranged from 73 percent in the 
District of Columbia to 85 percent in New York City. 
In public schools across the nation, 85 percent of 
students performed at or above the Basic level. In 
central city schools, 81 percent performed at or above 
the Basic level. 

Results for grade 8 

■ At grade 8, the percentages of students performing at 
or above Proficient ranged from 10 percent in Atlanta 
and the District of Columbia to 19 percent in Houston. 
Thirty percent of eighth-graders in public schools in 
the nation and 22 percent in central city schools 
performed at or above the Proficient level. 

■ The percentage of students performing at or above 
Proficient was higher for the nation than for any of 
the five urban districts reported, and higher for 
central cities than for all urban districts except 
Houston, where no significant difference was de- 
tected. The percentages of eighth-graders performing 
at or above Proficient in Chicago and Houston were 
not found to differ significantly from each other, and 
both were higher than the comparable percentages in 
the District of Columbia and Atlanta. 

■ The percentages of eighth-graders performing at or 
above Basic ranged from 64 percent in Los Angeles to 
74 percent in Houston. Eighty-four percent of eighth- 
graders in public schools in the nation and 77 per- 
cent in central city public schools performed at or 
above the Basic level. 



Results for Student Subgroups 

In addition to providing average scores and achievement 
levels for the nation, for states, and, in this report, for 
districts, NAEP reports provide results for subgroups of 
students defined by various background and contextual 
characteristics (e.g., gender, eligibility for free/reduced-price 
lunch, and level of parents’ education). In this report, 
performance results for subgroups are reported primarily as 
comparisons of district average scores with the comparable 
average scores in central cities. 

Gender 

Results for grade 4 

■ No statistically significant difference was detected 
between the average scores of male or female fourth- 
grade students in Houston and New York City and 
the average scores of their counterparts in the central 
city public schools. Average scores for fourth-grade 
male and female students in Atlanta, Chicago, the 
District of Columbia, and Los Angeles were lower 
than the average scores for their counterparts in 
central city schools. 

■ Female fourth-graders had higher average scores than 
male fourth-graders in each of the urban districts. 

Results for grade 8 

■ The average score for eighth-grade female students in 
Houston was not found to be significantly different 
from that of their counterparts in the central city 
public schools. In Atlanta, Chicago, the District of 
Columbia, and Los Angeles, the average scores for 
both male and female eighth-graders were lower than 
the average scores for their counterparts in central 
city schools. 

■ In all reported districts, female students had higher 
average writing scores than male students. 

Race/ethnicity 

In each of the urban districts assessed, Black or Hispanic 
students constituted the majority or the largest racial/ethnic 
group. This distribution differs from that for the national 
writing assessment, in which White students constituted a 
majority — 60 percent of the fourth-grade sample and 64 
percent of the eighth-grade sample. Black students made up 
more than four-fifths of the samples at both grades in 
Atlanta and the District of Columbia and nearly half at both 
grades in Chicago. Hispanic students made up about two- 
thirds of the Los Angeles samples at both grades and about 
half of the fourth-graders and more than half of the eighth- 
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graders in Houston. In New York City, more than two-fifths 
of the fourth-graders were Hispanic and just under a third 
were Black. 

Results for grade 4 

■ In the five urban districts in which a reliable com- 
parison could be made, White fourth-graders had 
higher average scores than their Black and Hispanic 
counterparts. 

■ Black students in grade 4 in Houston and New York 
City had higher average scores than their counter- 
parts in the central city public schools. Black fourth- 
grade students in Atlanta, Chicago, and Los Angeles 
had average scores not found to differ significantly 
from their counterparts in central cities. In the 
District of Columbia, Black fourth-graders had an 
average score lower than that of their counterparts in 
central cities. 

■ No significant difference was detected between the 
average score for Hispanic fourth-graders in four of 
the five districts in which a reliable comparison could 
be made and Hispanic fourth-graders’ average score 
in central cities. The average score for Hispanic 
fourth-graders in Los Angeles was lower than that in 
central cities taken as a whole. 

■ Average scores for White fourth-grade students in 
Atlanta, the District of Columbia, and New York City 
were higher than the average score for White fourth- 
grade students in central cities. 

■ Average scores for Asian/Pacific Islander students in 
Los Angeles and New York City were not found to be 
significantly different from the average score for their 
counterparts in central cities. 

Results for grade 8 

■ White eighth-graders had higher average scores than 
Black eighth-graders in every reported district except 
the District of Columbia, where the sample size was 
insufficient to permit a reliable comparison. White 
students at grade 8 also had higher average scores 
than Hispanic students in Chicago, Houston, and Los 
Angeles. 

■ The average score for Black eighth-grade students in 
the District of Columbia was lower than that for 
Black eighth-grade students in the central city public 
schools, and no significant difference was detected 
between the average score for Black students in any 



of the other four districts and the national average 
score for Black students in central cities. 

■ Average scores for Hispanic students were not found 
to differ significantly between the districts and the 
national average for central cities, except in Los 
Angeles, where Hispanic students had a lower 
average score than their counterparts in central cities. 

■ The average score for White eighth-grade students in 
Houston was higher than that of White students in 
the central city schools, while the average score in 
Los Angeles was lower. 

Eligibility for free/reduced-price lunch 

The National School Lunch Program providing free/ 
reduced-price lunch is administered by the U.S. Department 
of Agriculture (USDA) for children near or below the 
poverty line. Eligibility is determined by the USDA’s Income 
Eligibility Guidelines ( http://www.fns.usda.gov/cnd/ 
governance/notices/IEGs/IEGs.htm ). 

Results for grade 4 

■ At grade 4, rates of student eligibility for free/ 
reduced-price lunch ranged from 70 percent in New 
York City to 89 percent in Chicago. 

■ Fourth-grade students eligible for free/reduced-price 
lunch had lower average scores than those not 
eligible in every district except Los Angeles and New 
York City, where no significant difference between 
the two eligibility categories was detected. 

■ Fourth-grade students eligible for free/reduced-price 
lunch in New York City had a higher average score 
than the national average score for their counterparts 
in central city public schools, while students in 
Chicago and the District of Columbia had lower 
average scores than their eligible counterparts in 
central city schools. 

■ The average scale score for ineligible students at 
grade 4 in the District of Columbia was lower than 
the national average score for ineligible students in 
central city schools. 

Results for grade 8 

Because the available data for eligibility for eighth-graders 
in Los Angeles did not meet reporting standards, no 
information related to eligibility is reported for this segment 
of the sample. 
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■ At grade 8, the percentages of eligible students 
ranged from 67 percent in the District of Columbia to 
84 percent in Chicago. 

■ Students at grade 8 who were not eligible for free/ 
reduced-price lunch had a higher average score than 
eligible students in every district where the data were 
sufficiently reliable for significance testing. 

■ At grade 8, both those students eligible for free/ 
reduced-price lunch and those not eligible in Atlanta 
and the District of Columbia had lower average 
scores than their counterparts in the central city 
public schools. 

Parents' highest level of education 

Eighth-grade students who participated in the Trial Urban 
District Assessment were asked to indicate the highest level 
of education their parents had completed. Five response 
options — did not finish high school, graduated from high 
school, some education after high school, graduated from 
college, or “I don’t know” — were offered. 

■ In all five districts, lower percentages of students 
reported that their parents had graduated from 
college than in the national public school sample. 

The percentages of students who reported that their 
parents did not graduate from high school were 
higher in Chicago, Houston, and Los Angeles than in 
the nation. 

■ Atlanta and the District of Columbia had the highest 
percentages of students who reported that at least 
one parent had graduated from college (35 and 37 
percent, respectively). These percentages were 



significantly higher than those in Houston and Los 
Angeles. Atlanta also had a higher percentage of 
students reporting parents with some education after 
high school than all the other districts. Houston and 
Los Angeles had the highest percentages of students 
reporting parents who did not finish high school 
(22 and 18 percent, respectively). 

■ Average scores in all districts except Houston were 
lower for students who reported a college graduate 
parent than the national average score for their 
counterparts in the central city public schools. 

■ In Chicago and Houston, no statistically significant 
difference was detected between the average score of 
students with parents who did not finish high school 
and the average score of their counterparts in the 
central city schools, while the average score of these 
students in Atlanta, the District of Columbia, and Los 
Angeles was lower than the national average score of 
their counterparts in the central city schools. 



Data source: The National Assessment of Educational Progress (NAEP) 
2002 Trial Urban District Writing Assessment and 2002 Writing 
Assessment. 

For technical information, see the complete report: 

Lutkus, A.D., Daane, M.C., Weiner, A.W., and Jin, Y. (2003). The Nation's 
Report Card: Writing 2002, Trial Urban District Assessment 
(NCES 2003-530). 

Author affiliations: A.D. Lutkus, M.C. Daane, A.W. Weiner, and Y. Jin, 
Educational Testing Service. 

For questions about content, contact Taslima Rahman 
( taslima.rahman@ed.gov ). 

To obtain the complete report (NCES 2003-530), call the toll-free 
ED Pubs number (877-433-7827) or visit the NCES Electronic Catalog 
(h ttp://nces.ed.gov/pubsearch ). 
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Writing 2002 

This article describes the Statistics in Brief reports of the same name. The sample survey data are from the National Assessment of Educational Progress 
(NAEP) 2002 Trial Urban District Reading Assessment and 2002 Trial Urban District Writing Assessment, as well as the national 2002 Reading Assess- 



ment and 2002 Writing Assessment. 

In 2002, the National Assessment of Educational Progress 
(NAEP) conducted assessments of the reading and writing 
performance of public school students in several urban 
school districts. The NAEP 2002 Trial Urban District 
Assessment provided, for the first time, district-level results 
for Atlanta City, Chicago School District 299, Houston 
Independent School District, Los Angeles Unified, and New 
York City Public Schools. Results for the District of Colum- 
bia, which in 2002 and previous years participated in the 
NAEP state-level assessments, are reported along with 
results for the other urban districts. All the urban districts 
participated voluntarily in the assessments. 



In addition to the main reports on the Trial Urban District 
Assessment in reading and in writing (summarized earlier 
in this issue of the Quarterly ), each district receives custom- 
ized overviews of its results. For each subject at each grade 
level, a one-page Trial Urban District Assessment Snapshot 
Report summarizes the performance of public school 
students in the district. Each snapshot report includes 
overall results (average scale scores and percentages of 
students at each NAEP achievement level), results for 
various subgroups and subgroup score gaps, and average 
scale scores at selected percentiles. By way of example, 
figure A shows the snapshot report on the reading perfor- 
mance of New York City fourth-graders (the report was 
reduced to 80 percent of actual size in order to fit into this 
article). 



Data source: The National Assessment of Educational Progress (NAEP) 2002 Trial Urban District Reading Assessment and 2002 Trial Urban District Writing 
Assessment, as well as the national 2002 Reading Assessment and 2002 Writing Assessment. 

For technical information, see the following reports: 

Lutkus, A.D., Weiner, A.W., Daane, M.C., and Jin, Y. (2003). The Nation's Report Card: Reading 2002, Trial Urban District Assessment (NCES 2003-523). 

Lutkus, A.D., Daane, M.C., Weiner, A.W., and Jin, Y. (2003). The Nation's Report Card: Writing 2002, Trial Urban District Assessment (NCES 2003-530). 

For questions about content, contact Taslima Rahman ( taslima.rahman@ed.gov ). 

To obtain a district snapshot report (NCES 2003-534 for reading or NCES 2003-535 for writing), call the toll-free ED Pubs number (877-433-7827) or 
visit the NCES Electronic Catalog ( http://nces.ed.gov/pubsearch ). 
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Figure A. Sample Trial Urban District Assessment Snapshot Report on reading performance at grade 4 (shown at 80 
percent of actual size): New York City Public Schools, 2002 
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Invited Commentary: NAEP’s Trial Urban District Assessment: An Experiment 
Worth the Effort 

— Michael Casserly, Executive Director, Council of the Great City Schools 

This commentary represents the opinions of the author and does not necessarily reflect the views of the National Center for Education Statistics. 



Background 

Results from the 2002 Trial Urban District Assessment 
(TUDA) in reading and writing were released last summer, 
presenting the nation with its first look at how some of its 
largest urban public school systems fared on the National 
Assessment of Educational Progress (NAEP). Atlanta, 
Chicago, Houston, Los Angeles, and New York City volun- 
teered for the district-level testing after the Council of the 
Great City Schools approached the National Assessment 
Governing Board in the fall of 2000 with a proposal to allow 
cities to get NAEP results similar to those that participating 
states have been receiving for many years. Results from the 
District of Columbia, which participates in NAEP’s state- 
level assessments, were analyzed along with those from the 
other cities. 

The bid for district-level results was a surprise to many, 
coming as it did from the nation’s big-city school systems, 
but the Council made it for a number of reasons: 

■ First, the nation’s urban school systems wanted to 
make it crystal clear once again that they are fully 
committed to the highest academic standards for 
their students. 

■ Second, they wanted a way of gauging their progress 
and evaluating their reforms in ways that the nation’s 
current state-by-state testing system does not allow. 

■ Third, they wanted to be able to compare themselves 
against other school systems facing many of the same 
challenges. 

Establishing a Benchmark 

There were realistic expectations, of course, about what the 
initial results would show. But the cities were determined to 
establish a benchmark by which they could measure their 
progress. That benchmark was set in the 2002 reading and 
writing TUDA. 

Overall results 

The 2002 TUDA results did not present many major 
surprises. Reading performance in the participating cities 
was below the national public school average (table 1). 

The percentage of urban fourth-graders scoring at or above 
the Basic level of achievement ranged from 48 percent in 



Houston to 31 percent in the District of Columbia, com- 
pared with 62 percent nationally. The percentage of eighth- 
graders scoring at or above the Basic level ranged from 62 
percent in Chicago to 42 percent in Atlanta, compared with 
74 percent nationally. (Data on eighth-graders in New York 
City were not available.) 

Writing results indicated that the cities did somewhat 
better, with two districts having average fourth-grade scores 
that were statistically indistinguishable from the national 
average (table 2). The percentage of fourth-graders scoring 
at or above Basic ranged from 85 percent in New York City 
to 73 percent in the District of Columbia, compared with 85 
percent nationally. The percentage of eighth-graders scoring 
at or above Basic ranged from 74 percent in Houston to 64 
percent in Los Angeles, compared with 84 percent nation- 
ally. (Again, data on eighth-graders in New York City were 
not available.) 

Student characteristics 

Background data collected as part of the assessment also 
showed how different demographically the TUDA cities are 
from the rest of the nation. Tables 1 and 2 indicate that the 
majority of students in these cities, unlike the nation as a 
whole, were eligible for free or reduced-price school lunch 
(intended for children near or below the poverty line). The 
proportion of limited-English-proficient (LEP) students in 
some of the cities was also considerably higher than the 
national average. Finally, each of the TUDA cities had 
enrollments that were predominantly Black or Hispanic, 
a situation that is significantly different from national 
averages (not shown in tables 1 and 2). 

Subgroup results 

The initial TUDA results also provided data on achievement 
by race/ethnicity, eligibility for free or reduced-price school 
lunch, and other student characteristics, giving analysts and 
practitioners a better sense of how the performance of 
subgroups in the cities compares with that of the same 
subgroups nationally. 

The reading results, for instance, indicated that White 
students, in general, scored at about the same level in 
TUDA cities as White students nationally. Black students 
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Table 1 . Percentage of students eligible for free/reduced-price lunch, percentage of limited-English-proficient (LEP) students, and NAEP reading 
results, grade 4 and grade 8 public schools: By urban district, 2002 









At grade 4 








Percent of students 




NAEP reading results 




Free/reduced- 
price lunch 


LEP 


Average 

scale 

score 


At or 
above 
Basic 


Percent of students 

At or 
above 

Proficient At Advanced 


Atlanta 


74 


4 


195* 


35* 


12* 


3* 


Chicago 


88 


19 


193* 


34* 


11* 


2* 


Houston 


72 


36 


206* 


48* 


18* 


3* 


Los Angeles 


79 


46 


191* 


33* 


11* 


2* 


New York City* 


73 


11 


206* 


47* 


19* 


5 


District of Columbia 


78 


7 


191* 


31* 


10* 


2* 


National average 


43 


9 


217 


62 


30 


6 








At grade 8 










Percent of students 




NAEP reading results 














Percent of students 








Average 


At or 


At or 






Free/reduced- 




scale 


above 


above 






price lunch 


LEP 


score 


Basic 


Proficient 


At Advanced 


Atlanta 


76 


1 


236* 


42* 


8* 


#* 


Chicago 


84 


8 


249* 


62* 


15* 


1 


Houston 


68 


16 


248* 


59* 


17* 


1* 


Los Angeles 


4 


30 


237* 


44* 


10* 


#* 


New York City 


4 


4 


4 


4 


4 


4 


District of Columbia 


68 


5 


240* 


48* 


10* 


#* 


National average 


34 


6 


263 


74 


31 


2 



#Rounds to zero. 

^Although the New York City response rate was deemed sufficient for reporting at grade 4, the target rate specified in the National Assessment of Educational 
Progress (NAEP) guidelines was not met. At grade 8, the New York City response rate did not meet reporting standards. At grade 8, Los Angeles data on free/ 
reduced-price lunch did not meet reporting standards. 

Significantly different from the national average for public schools. 

SOURCE: Lutkus, A.D., Weiner, A.W., Daane, M.C., and Jin, Y. (2003). The Nation's Report Card: Reading 2002, Trial Urban District Assessment (NCES 2003-523). U.S. 
Department of Education. Washington, DC: National Center for Education Statistics. 



and students eligible for free or reduced-price lunch, on the 
other hand, generally had lower average scores in TUDA 
cities than nationally. And Hispanic students in TUDA cities 
had average scores about the same as or lower than the 
national average for Hispanic students. 

In a number of cities, however, Black, Hispanic, and poor 
students had reading scores that equaled those of their 
racial/ethnic and income peers nationally. For example, 
Black fourth- and eighth-graders in Houston had average 
scores that did not differ significantly from those of their 
Black peers nationally. The same was true of Black fourth- 
graders in New York City and Black eighth-graders in 
Chicago. At both the fourth and the eighth grades, Hispanic 
students in three TUDA cities scored about the same as 
their Hispanic peers nationally. At the fourth grade in New 



York City and Houston, and at the eighth grade in Chicago, 
students eligible for free or reduced-price school lunch 
posted average scores that were statistically indistinguish- 
able from those of eligible students nationally. 

A number of interesting results emerged from the writing 
assessment as well. White students in TUDA cities generally 
posted average scores that were about the same as or above 
the national average for White students. Black fourth- and 
eighth-graders in Houston and Black fourth-graders in New 
York City outscored their Black peers nationally, while Black 
students in other TUDA cities scored lower than or about 
the same as Black students nationally. In almost all cases, 
the average scores of Hispanic students in TUDA cities were 
about the same as the average for Hispanic students nation- 
ally. Students eligible for free or reduced-price lunch often 
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Table 2. Percentage of students eligible for free/reduced-price lunch, percentage of limited-English-proficient (LEP) students, and NAEP writing 
results, grade 4 and grade 8 public schools: By urban district, 2002 









At grade 4 








Percent of students 






NAEP writing results 












Percent of students 








Average 


At or 


At or 




Free/reduced- 




scale 


above 


above 




price lunch 


LEP 


score 


Basic 


Proficient At Advanced 


Atlanta 


73 


4 


140* 


77* 


13* 1 


Chicago 


89 


19 


138* 


76* 


12* # 


Houston 


72 


33 


148 


81 


23 2 


Los Angeles 


78 


49 


141* 


77* 


16* 1* 


New York City* 


70 


13 


153 


85 


27 2 


District of Columbia 


78 


7 


135* 


73* 


11* 1* 


National average 


43 


9 


153 


85 


27 2 








At grade 8 








Percent of students 






NAEP writing results 












Percent of students 








Average 


At or 


At or 




Free/reduced- 




scale 


above 


above 




price lunch 


LEP 


score 


Basic 


Proficient At Advanced 


Atlanta 


74 


1 


130* 


68* 


10* # 


Chicago 


84 


8 


136* 


72* 


16* 1 


Houston 


68 


18 


138* 


74* 


19* 1 


Los Angeles 


4 


30 


128* 


64* 


11* # 


New York City 


4 


4 


4 


4 


4 4 


District of Columbia 


67 


5 


128* 


66* 


10* # 


National average 


34 


6 


152 


84 


30 2 



#Rounds to zero. 

^Although the New York City response rate was deemed sufficient for reporting at grade 4, the target rate specified in the National Assessment of Educational 
Progress (NAEP) guidelines was not met. At grade 8, the New York City response rate did not meet reporting standards. At grade 8, Los Angeles data on free/ 
reduced-price lunch did not meet reporting standards. 

Significantly different from the national average for public schools. 

SOURCE: Lutkus, A.D., Daane, M.C., Weiner, A.W., and Jin, Y. (2003). The Nation's Report Card: Writing 2002, Trial Urban District Assessment (NCES 2003-530). U.S. 
Department of Education. Washington, DC: National Center for Education Statistics. 



scored lower in TUDA cities than nationally. In three cities, 
however, scores of eligible fourth-graders were about the 
same as the national average for eligible fourth-graders. 

Gaps between different subgroups 

The 2002 TUDA results provided fresh data on the size of 
the achievement gaps within and across the participating 
cities and how they stack up to gaps nationwide, tn some 
cities, gaps between the achievement of certain subgroups 
(e.g., White students and Black students) differed from the 
gaps in the nation as a whole. In other cities, the gaps 
reflected national averages. At grade 4, for example, the 
results showed that gaps between the average reading and 
writing scores of White and Black students in Atlanta and 
the District of Columbia were wider than the national gaps. 



But in Houston, Los Angeles, and New York City, the 
fourth-grade reading and writing gaps between Black and 
White students were about the same as those nationally. 

Promoting Improvement in Big-City Schools 

TUDA results are important to cities and to others inter- 
ested in the reform and improvement of big-city schools. 
They provide much-needed data on urban school perfor- 
mance that allows comparisons in ways that the nation’s 
state-by-state assessment system cannot (Council of the 
Great City Schools 2003). TUDA will also provide valuable 
insights into the performance of big-city schools over time.* 



*After this commentary was written, results from the 2003 TUDA were released 
(Lutkus and Weiner 2003a and 2003b). Reading results for 2003 are now available for 
the cities that participated in the 2002 TUDA, as well as for four additional cities. 
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But the promise of TUDA stretches well beyond its ability to 
determine whether big-city schools are improving. Its real 
importance rests in its power to shed light eventually on 
which reforms are working and which ones are not. TUDA 
presents the cities and researchers interested in analyzing 
patterns in urban schools with a chance to establish a long- 
term, informal field experiment where each city’s reforms 
can be gauged against others using a common benchmark. 

This kind of research is not commonly done at present, 
however, in part because there is no common metric on 
which cities can be measured and in part because it is hard 
to do. The overwhelming majority of research on urban 
education is devoted to efforts to turn around individual 
schools or pockets of schools; efforts to assess the effects of 
discrete programs or program strategies; and efforts to 
measure student demographic and background characteris- 
tics and the effects of these characteristics on performance. 

Some observers have argued that the way to bring about 
wide-scale improvement in urban education is to take what 
has been learned about each individual school or program 
and apply it systemwide. This approach has had limited 
success, however, because it does not help school practitioners 
sort through the numerous programs or leverage systemwide 
change. Very few reforms or research studies, in fact, have 
focused on how one improves an individual system or how 
one accelerates improvements across many cities. 

Public impatience and No Child Left Behind will no longer 
allow the nation’s major city school districts to improve 
themselves using a school-by-school strategy. Time will not 
allow such a piecemeal approach. Instead, the public and 
the new law are demanding improvements in city schools 
writ large. And city school leaders, for their part, need better 
research on strategies that will help improve student 
performance systemwide, not just in some schools or for 
some students. 

The lack of solid research on how to turn around public 
school systems in big cities has been frustrating both for the 
urban school districts themselves and for others. There has 
been little way, for instance, to determine which big-city 
school systems were improving student achievement the 
most. Consequently, there has been little way to determine 
for sure what those faster improving school districts were 
doing to get their gains that others were not doing (Snipes, 
Doolittle, and Herlihy 2002). 



If we had comparable data to tell us, for example, that Black 
students in Atlanta were improving faster in reading than 
Black students in other cities, then we would know where 
to look for promising practices about how to accelerate the 
reading skills of Black students. If city school systems with 
varying governance structures were improving at about the 
same rates, we might conclude that governance structure 
was not the critical ingredient in the gains. If academic 
progress in one set of cities extended beyond 3 years, then 
we might be able to say something about how to sustain 
the effects of reform. If cities of similar composition were 
making progress in some states but not in others, then we 
might be able to ask better questions about the role of states 
in boosting urban performance. If faster improving cities 
were using a single reading curriculum and slower moving 
cities were relying on individual-school reform models, 
then urban school leaders might have a better sense about 
how to structure their instructional programs. If faster 
improving cities had spending levels that were below their 
respective statewide averages, then researchers and advo- 
cates might have a better understanding about the role of 
money in improving urban schools. 

Asking questions of this kind requires a more institutional, 
systemic, and organizational outlook about the behavior of 
urban school districts than many in the reform movement 
or the research community have hitherto shown. There are 
only a handful of people nationwide, in fact, who think in 
terms like these. Yet, answers to these and similar questions 
are critical if urban education is to give the public the 
results that it wants. Ultimately, urban school leaders will 
never be able to get gains “at scale” unless they think at 
scale. TUDA allows that to begin happening. 

TUDA will not answer questions about improving big-city 
school systems on its own, of course. There are too many 
moving gears in urban schools for us to be certain about 
anything. But TUDA gives urban educators and others 
another tool with which to ask better questions. 

The 2002 TUDA results are only a starting point. Over 
time and with a larger pool of participating cities, critical 
questions about how to leverage improvements in big-city 
school districts can be answered and acted upon. Urban 
school leaders are determined to raise student performance 
in their communities. TUDA will help let them know if they 
are succeeding. More importantly, TUDA can help them 
succeed faster. It is an experiment worth pursuing. 
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The Nation’s Report Card: Writing 2002 

Hilary R. Persky, Mary C. Daane, and Yingjin 

This article was originally published as the Executive Summary of the Statistical Analysis Report of the same name. The sample survey data are from the 
National Assessment of Educational Progress (NAEP) 1998 and 2002 Writing Assessments. 



The National Assessment of Educational Progress (NAEP) is 
an ongoing nationally representative sample survey of 
student achievement in core subject areas. Authorized by 
Congress and administered by the National Center for 
Education Statistics (NCES) within the Institute of Educa- 
tion Sciences of the U.S. Department of Education, NAEP 
regularly reports to the public on the educational progress 
of fourth-, eighth-, and twelfth-grade students. 

This report presents the results of the NAEP 2002 Writing 
Assessment for the nation at grades 4, 8, and 12 and for 
participating states and other jurisdictions at grades 4 



and 8. Assessment results are described in terms of average 
writing score on a 0-300 scale and in terms of the percent- 
age of students attaining each of three achievement levels: 
Basic, Proficient, and Advanced. 

The achievement levels are performance standards adopted 
by the National Assessment Governing Board (NAGB) as 
part of its statutory responsibilities. The achievement levels 
are a collective judgment of what students should know and 
be able to do for each grade tested. As provided by law, 
NCES, upon review of a congressionally mandated evalua- 
tion of NAEP, determined that the achievement levels are to 
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be considered developmental and should be interpreted 
with caution. However, both NCES and NAGB believe that 
these performance standards are useful for understanding 
trends in student achievement. They have been widely used 
by national and state officials as a common yardstick of 
academic performance. 

The results presented in this report are based on representa- 
tive samples of students for the nation and for participating 
states and other jurisdictions. Approximately 276,000 
students from 11,000 schools were assessed. The national 
results reflect the performance of students attending both 
public and nonpublic schools, while the results for states 
and other jurisdictions reflect only the performance of 
students attending public schools. Information about writing 
achievement for students in selected urban school districts 
is presented in The Nation’s Report Card: Writing 2002, Trial 
Urban District Assessment (Lutkus et al. 2003), summarized 
earlier in this issue of the Quarterly. 

In addition to providing average scores and achievement- 
level performance in writing for the nation and states and 
other jurisdictions, this report provides results for sub- 
groups of students defined by various background charac- 
teristics. A summary of major findings from the NAEP 2002 
assessment is presented below. Comparisons are made to 
national results from the 1998 assessment. The NAEP 1998 
Writing Assessment was not administered at the state level 
at grade 4; therefore, state-level comparisons are presented 
only for grade 8. 

Overall Writing Results for the Nation and the 
States 

Writing results for the nation 

■ Students’ average scores on the NAEP writing 
assessment increased between 1998 and 2002 at 
grades 4 and 8 (figure A) . However, no significant 
change was detected in the performance of twelfth- 
graders between the 2 assessment years. 

■ Fourth-grade writing scores at the 10th to the 90th 
percentiles increased between 1998 and 2002. This 
means that the performance of higher-, middle-, and 
lower-performing students improved between the 

2 years. Gains were observed among the middle- and 
higher-performing students at grade 8. At grade 12, 
only the score at the 90th percentile increased between 
1998 and 2002, while scores at the 10th and 25th 
percentiles were lower in 2002. 



■ In 2002, between 24 and 31 percent of the students 
in each of the three grades performed at or above the 
Proficient level. Fourth- and eighth-graders made 
overall gains between 1998 and 2002 in reaching the 
Proficient level. There was no significant change 
detected in the percentage of twelfth-graders at or 
above Proficient, however, the percentage of twelfth- 
graders at or above Basic decreased over the period. 

Writing results for the states and other jurisdictions 

Results from the 2002 assessment are reported for 48 states 
and other jurisdictions at grade 4, and 47 states and other 
jurisdictions at grade 8. Results are reported only for public 
school students at the state level. 

At grade 4 

■ In 2002, fourth-grade average scores were higher 
than the national average score in 17 jurisdictions, 
and lower than the national average in 22 jurisdic- 
tions (figure B) . 

■ Connecticut, Massachusetts, and Delaware were 
among the highest performing jurisdictions at grade 
4. The average writing scores in Connecticut and 
Massachusetts were higher than in any of the other 
participating jurisdictions. Massachusetts was only 
outperformed by Connecticut. Students in Delaware 
were only outperformed by students in Connecticut 
and Massachusetts and had higher scores than 
students in the other participating jurisdictions 
except New York. 

At grade 8 

■ Of the 36 jurisdictions that participated in both the 
1998 and 2002 eighth-grade writing assessment, 

16 showed score increases in 2002 and none showed 
a significant decrease. 

■ The percentage of eighth-graders at or above Profi- 
cient increased in 17 jurisdictions and decreased in 
1 jurisdiction between the 2 assessment years. 

■ Connecticut, Department of Defense domestic and 
overseas schools, Massachusetts, and Vermont were 
among the highest performing jurisdictions at grade 8. 

National and State Writing Results for 
Student Subgroups 

In addition to overall results for the nation and for the 
states and other jurisdictions, NAEP reports on the perfor- 
mance of various subgroups of students. Observed differ- 
ences between student subgroups in NAEP writing performance 
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Figure A. Average writing scale scores, grades 4, 8, and 1 2: 1 998 and 2000 




^Significantly different from 2002. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, 
National Assessment of Educational Progress (NAEP), 1 998 and 2002 Writing 
Assessments. (Based on figure 2.1 on p. 1 8 of the complete report from which this 
article is excerpted.) 



most likely reflect the interaction of a range of socioeco- 
nomic and educational factors not addressed in this report 
or by NAEE 

National results 
Gender 

■ The average scores of male and female fourth- and 
eighth-graders were higher in 2002 than in 1998; 
however, at grade 12, the average scores for male 
students declined. 

■ The percentages of female students performing at or 
above Proficient increased between 1998 and 2002 at 
all three grades, and the percentage of male students 
performing at or above Proficient increased at grades 
4 and 8. 

■ In 2002, female students had higher average scores 
than male students at all three grades. 

■ In 2002, females outperformed males, on average, by 
17 points at grade 4, 21 points at grade 8, and 25 points 
at grade 12. The decline in the average score for male 
twelfth-graders between 1998 and 2002 resulted in 



an increase in the gap between male and female 
students. 

Race/ethnicity 

■ At grades 4 and 8, White, Black, and Hispanic 
students had higher average writing scores in 2002 
than in 1998 (figure C). 

■ The percentages of students performing at or above 
Proficient increased between 1998 and 2002 among 
White, Black, Hispanic, and Asian/Pacific Islander 
students at grade 4 and among White, Black, and 
Hispanic students at grade 8. 

■ At grade 4, Asian/Pacific Islander students outper- 
formed all other groups in 2002, and White students 
outperformed Black, Hispanic, and American Indian/ 
Alaska Native students. At grade 8, White and Asian/ 
Pacific Islander students scored higher, on average, 
than Black, Hispanic, and American Indian/Alaska 
Native students. At grade 12, White and Asian/Pacific 
Islander students scored higher, on average, than 
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Figure B. Comparison of state and national public school average writing scores, grades 4 and 8: 2002 



Grade 4 




Jurisdiction had higher average scale 
score than nation. 



T1 Jurisdiction was not found to be 
significantly different from nation in 
average scale score. 

~j Jurisdiction had lower average scale 
score than nation. 



Jurisdiction did not meet minimum 
participation rate guidelines. 



Jurisdiction did not participate in the 
NAEP 2002 Writing State Assessment. 



Grade 8 




American 

Samoa 






1 Department of Defense Domestic Dependent Elementary and Secondary Schools, 
department of Defense Dependents Schools (overseas). 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2002 Writing Assessment. (Originally published 
as figures 2.4 and 2.5 on pp. 25 and 26 of the complete report from which this article is excerpted.) 



Black and Hispanic students, and Hispanic students 
had higher scores than Black students. 

In 2002, the score gap between White and Black 
fourth-graders was smaller than in 1998. 



Eligibility for free/reduced-price lunch 

The program providing free/reduced-price lunch is adminis- 
tered by the U.S. Department of Agriculture (USDA) for 
children near or below the poverty line. Eligibility is 



30 



NATIONAL CENTER FOR EDUCATION STATISTICS 







The Nation's Report Card: Writing 2002 



Figure C. Average writing scale scores, by race/ethnicity, grades 4, 8, and 1 2: 1 998 and 2002 




^Significantly different from 2002. 

NOTE: Italicized scale score values indicate that two or more groups had the same average scale score when rounded: Grade 4, 1 998: Black and American Indian/Alaska Native 
students (the 1 998 score was significantly different from 2002 only for Black students); Grade 8, 1 998: Black and Hispanic students (the 1 998 scores were significantly different from 
2002 for both Black and Hispanic students); Grade 8,2002: White and Asian/Pacific Islander students, and Hispanic and American Indian/Alaska Native students. Quality control 
activities and special analysis raised concerns about the accuracy and precision of grade 1 2 American Indian/Alaska Native data in 2002. As a result, they are omitted from this 
report. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1 998 and 2002 Writing Assessments. (Taken 
from figure 3.3 on p.46 of the complete report from which this article is excerpted.) 



determined by the USDA’s Income Eligibility Guidelines 
( http://www.fns.usda.gov/cnd/governance/notices/IEGs/ 
IEGs.htm ). 

■ Average fourth- and eighth-grade writing scores in 
2002 were higher than in 1998 for students who were 
eligible for free/reduced-price lunch, as well as for 
those who were not eligible. 

■ The percentages of fourth- and eighth-graders at or 
above Proficient were higher in 2002 than in 1998 for 
students who were eligible and those who were not 
eligible for free/reduced-price lunch. 

■ In 2002, the average writing score for students who 
were eligible for free/reduced-price lunch was lower 
than that of students who were not eligible at all 
three grades. 

Title I participation 

Title I is a federally funded program that provides educa- 
tional services to children who live in areas with high 



concentrations of low-income families. Due to recent 
changes in how the program is administered, comparisons 
to previous assessment year results are not available. 

■ In 2002, students at all three grades who attended 
schools that participated in Title I had lower average 
writing scores than students who attended schools 
that did not participate in Title I. 

Parents' level of education 

■ There was a positive relationship between higher 
levels of parental education as reported by students 
and student achievement: for both eighth- and twelfth- 
graders, the higher the parental education level, the 
higher the average writing score. (Information about 
parental education was not collected at grade 4.) 

Type of school 

■ The average writing scores for fourth- and eighth- 
grade public school students were higher in 2002 
than in 1998. 
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■ In 2002, at all three grades, students who attended 
nonpublic schools had higher average writing scores 
than students who attended public schools. At grade 8, 
students who attended Catholic schools had higher 
scores than those attending other nonpublic schools. 

Type of school location 

■ Students in urban fringe schools had higher average 
writing scores than their peers in central city schools 
and rural schools at all three grades. Fourth- and 
eighth-grade students in rural schools had higher 
scores than their peers in central city schools, while 
the reverse was true at grade 12. 

State and other jurisdiction results 

Gender 

■ At grade 8, average scores were higher in 2002 than 
in 1998 for both male and female students in 12 
jurisdictions, for female students only in 1 jurisdic- 
tion, and for male students only in 2 jurisdictions. 

■ In 2002, females had higher average scores than 
males in all the participating jurisdictions at both 
grades 4 and 8. 

Race/ethnicity 

■ At grade 8, average scores increased between 1998 
and 2002 for White students in 15 jurisdictions, 
for Black students in 9 jurisdictions, for Hispanic 
students in 4 jurisdictions, and for students classified 
as Other in 1 jurisdiction. 



■ Score increases were observed for two or more racial/ 
ethnic subgroups of eighth-graders in the following 
jurisdictions: Arkansas, Delaware, Florida, Louisiana, 
Maryland, Missouri, North Carolina, South Carolina, 
and Washington. 

Eligibility for free/reduced-price lunch 

■ At grade 8, average scores increased between 1998 
and 2002 for both those students who were eligible 
for free/reduced-price lunch and those who were not 
eligible in 1 1 jurisdictions, only for eligible students 
in 1 jurisdiction, and only for students who were not 
eligible in 4 jurisdictions. 
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This report examines patterns and trends in the vocational/ 
technical coursetaking of public high school graduates 
between 1982 and 1998. It updates and expands upon 
trends that were published in the National Center for 
Education Statistics (NCES) report Vocational Education in 
the United States: Toward the Year 2000 (Levesque et al. 
2000). Specifically, the current report includes trends in the 
participation of graduates based on their special and 
protected population status, including race/ethnicity, sex, 
disability status, English proficiency, and several measures 
of student academic achievement, as well as school 
urbanicity and school poverty level. The report analyzes 
these trends by examining high school transcripts for the 
graduating classes of 1982, 1990, 1992, 1994, and 1998. 1 
The analysis samples and variables used in the report are 
comparable across the survey years. The analysis focuses on 
public high school graduates who earned regular or honors 
diplomas. 2 

Transcripts provide information on the courses that public 
high school graduates took in grades 9 through 12. For 
simplicity’s sake, the report refers to this information as 
“high school coursetaking.” With the exception of a few 
tables that examine coursetaking in each grade (9 through 
12) separately, the report describes the cumulative course- 
work that graduates took in high school. The report is 
intended to accompany the N CES report Trends in High 
School Vocational/Technical Coursetaking: 1982-1998 
(Levesque 2003), which provides an in-depth examination 
of the vocational/technical coursetaking patterns of public 
high school graduates in general. 

'These transcript studies were conducted as part of the High School and Beyond 
Longitudinal Study of 1 980 Sophomores, "High School Transcript Study" (HS&B-So:80/ 
82) regarding 1 982 graduates; the National Education Longitudinal Study of 1 988 
(NELS:88/92), "Second Follow-up,Transcript Survey, 1992" regarding 1992 graduates; 
and the High School Transcript Studies (HSTS) of 1 990, 1 994, and 1 998 regarding 1 990, 
1 994, and 1 998 graduates, respectively. 

2 The HS&B and NELS studies excluded students with the most severe disabilities, 
where it was determined by school staff that these students were unable to complete 
the lengthy student questionnaires that were a part of these studies. In order to 
ensure comparability across the data sets, graduates with special education diplomas 
were excluded from the HSTS samples (Gifford et al. 1989;Tuma 1996). Thus, the 
samples used for this trend analysis were consistent with the population of public 
high school graduates, including students with disabilities, who earned regular or 
honors diplomas in each of the study years. In addition, there may be some minor 
coding differences between NELS and the other transcript data that may affect the 
data for 1992. See appendix C of the full report for more information. 



Terms Used in the Report 

The vocational/technical curriculum 

The NCES Secondary School Taxonomy (SST) classifies 
high school vocational/technical education into three dif- 
ferent curricula: specific labor market preparation, or 
“occupational education”; general labor market preparation; 
and family and consumer sciences education. Occupational 
education consists of courses that teach skills and knowl- 
edge required in a particular occupation or set of related 
occupations. General labor market preparation consists of 
courses that teach general employment skills that are not 
specific to one occupational area, such as basic typewriting/ 
keyboarding, introductory technology education, and career 
preparation and general work experience courses. Family 
and consumer sciences education consists of courses in- 
tended to prepare students for family and consumer roles 
outside the paid labor market. 3 For purposes of this report, 
trends focus on vocational/technical coursetaking overall 
and on occupational coursetaking. 

Although vocational/technical coursetaking is prevalent in 
high schools, students take varying amounts and types of 
these courses and take them for different purposes. This 
report emphasizes the coursetaking patterns of occupational 
concentrators because this group is a common focus of fed- 
eral and state accountability and research efforts for voca- 
tional/technical education (U.S. Department of Education 
2002; Silverberg et al. 2002). Occupational concentrators 
are graduates who earned 3.0 or more credits during high 
school in one of the following 10 broad occupational 
program areas: agriculture, business, marketing, health care, 
protective services, trade and industry, technology, food 
service and hospitality, child care and education, and 
personal and other services. In some cases, the report also 
examines trends in concentrating (earning 3.0 or more 
credits) in 18 narrow occupational program areas. 4 



3 Home economics-related courses that prepare students for the paid labor market are 
included under occupational education. 

4 These include agriculture, business services, business management, marketing, health 
care, protective services, construction, mechanics and repair, print production, 
materials production, other precision production, transportation, computer technol- 
ogy, communications technology, other technology, food service and hospitality, child 
care and education, and personal and other services. 
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Key population variables 

The Carl D. Perkins Vocational and Applied Technology 
Education Act Amendments of 1998 (1998 Perkins Act) 
defines “special populations” as follows: 

■ individuals with disabilities; 

■ individuals from economically disadvantaged 
families, including foster children; 

■ individuals preparing for occupations that are 
nontraditional for their gender; 

■ single parents, including single pregnant women; 

■ displaced homemakers; and 

■ individuals with other barriers to educational 
achievement, including individuals with limited 
English proficiency. 

The 1990 Perkins Act, which governed the second half of 
the period covered in this report (1990-1998), defined 
“special populations” fairly similarly, including individuals 
with handicaps, educationally and economically disadvan- 
taged individuals (including foster children), individuals of 
limited English proficiency, individuals who participate in 
programs designed to eliminate sex bias, and individuals in 
correctional institutions. 

In addition, the Office for Civil Rights (2001) in the U.S. 
Department of Education enforces federal statutes that 
prohibit discrimination in education programs and activities 
receiving federal financial assistance (such as Perkins Act 
funds) on the following bases: race, color, national origin, 
sex, disability, and age. 

To the extent possible, this report provides information on 
trends in the vocational/technical coursetaking of these 
special and protected populations, as well as their peers 
who were not members of these groups. To do so, the report 
uses the following categories. Measures were selected based 
on federal definitions, previous related research, and data 
availability. Data were provided only for those years and 
surveys that contained comparable variables. For the sake 
of readability when summarizing findings, the report uses 
the terms disadvantaged and advantaged to describe student 
groups on some of the key variables, as indicated below. 5 

Race/ethnicity. Includes the five categories of American 
Indian/Alaska Native; Asian/Pacific Islander; Elispanic; non- 
Hispanic Black; and non-Hispanic White. For simplicity’s 

5 ln a few cases, advantaged students were further classified as moderately advantaged 
and highly advantaged. The race/ethnicity and sex categories were not classified 
according to advantage, because the Perkins legislation did not make this distinction 
for these variables. 



sake, the text refers to Black and White graduates, although 
students in both of these groups were also non-Hispanic. 

Sex. Includes the two categories of male and female. 

Disability status (grade 12). Includes students who were 
reported to have a disability and students who were re- 
ported to have no disability as of grade 12. It should be 
remembered, however, that graduates with the most severe 
disabilities were excluded from the analysis due to survey 
constraints. Consequently, the disability status variable 
identifies students with and without disabilities among the 
population of public high school graduates who earned 
regular or honors diplomas. 6 For purposes of this analysis, 
students with disabilities were considered to be “disadvan- 
taged,” while students without disabilities were considered 
to be more “advantaged.” 

English proficiency (grade 12). Includes the two categories 
of limited English proficiency and English proficient. It is 
important to note that this variable describes students’ 
English language proficiency as of grade 12. For purposes of 
this analysis, graduates who had limited English proficiency 
in grade 12 were considered to be “disadvantaged,” while 
graduates who were English proficient in grade 12 were 
considered to be more “advantaged.” 

The report uses the following three measures of academic 
achievement: 7 

Grade point average (GPA). Calculated from grades re- 
corded in the transcript files, this variable has a range of 0.0 
to 4.0. It was not possible in some of the surveys to calcu- 
late GPA for academic courses only (a preferable measure of 
academic achievement), so overall GPA was used. GPA was 
collapsed into three categories: high GPA (greater than 3.5); 
mid-level GPA (2.0 to 3.5); and low GPA (less than 2.0). 

For purposes of this analysis, students with a GPA of less 
than 2.0 were considered to be “disadvantaged”; students 
with a GPA of 2.0 to 3.5 were considered to be “moderately 
advantaged”; and students with a GPA of greater than 3.5 
were considered to be “highly advantaged.” 



6 As of 1 998, about 31 percent of students with disabilities held special education 
diplomas and were excluded from the study. 

7 The final federal regulations to the 1 990 Perkins Act used grade point average to 
define academically disadvantaged individuals. The other two measures were 
suggested by previous research on whether vocational education has been a 
"dumping ground"for low academically achieving students (Boesel et al. 1 994). The 
1998 Perkins Act offered no additional guidance for identifying students with barriers 
to educational achievement, other than limited English proficiency. 
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Academic coursework completed. This variable describes 
whether students completed all low or all high mathemat- 
ics, science, and English courses, or some other combina- 
tion of mathematics, science, and English courses (mid- 
level or mixed academic coursetaking). Low and high 
mathematics, science, and English courses are defined in 
the glossary (appendix B) and technical appendix (appen- 
dix C) of the full report. For purposes of this analysis, 
students completing all low-level academic coursework 
were considered to be “disadvantaged”; students completing 
all high-level academic coursework were considered to be 
“highly advantaged”; and students completing mid-level or 
mixed academic coursework were considered to be “moder- 
ately advantaged.” 

Grade 9 mathematics. This variable identifies the math- 
ematics course a student took in grade 9. ft includes the 
three categories of high-level grade 9 mathematics (geom- 
etry or higher), mid-level grade 9 mathematics (pre-algebra 
or algebra 1), and low-level mathematics (no mathematics 
or mathematics courses below pre-algebra). It provides a 
measure of academic achievement before most of graduates’ 
coursework in vocational/technical education was taken 
and is therefore less confounded with that coursetaking 
than either GPA or academic coursework completed. For 
purposes of this analysis, students who took low-level 
mathematics in grade 9 were considered to be “disadvan- 
taged”; students who took mid-level grade 9 mathematics 
were considered to be “moderately advantaged” ; and 
students who took high-level grade 9 mathematics were 
considered to be “highly advantaged.” 

Although a student-level measure of socioeconomic status 
would have been preferable for this analysis, such a variable 
was not available from the 1990, 1994, and 1998 High 
School Transcript Studies (HSTS). Instead, the report uses 
the following two school-level variables as measures of 
economic status: 8 

School urbanicity. This variable describes the location of the 
school a graduate attended in the 12th grade and includes 
the three categories of urban, suburban, and rural. These 
categories are defined further in appendixes B and C of the 
full report. 



8 Section 421 of the 1990 Perkins Act included information on students in rural and 
urban areas in its identification of economically disadvantaged students.The final 
regulations to the 1 990 Perkins Act also included eligibility for the National School 
Lunch Program in the definition of this group. The 1998 Perkins Act provided no 
additional guidance on defining economically disadvantaged students. 



School poverty level. This variable describes the proportion 
of students in the school a graduate attended in the 12th 
grade who participated in the National School Lunch 
Program (NSLP). It includes the categories of high poverty 
(greater than 50 percent in NSLP) and low poverty (5 per- 
cent or less in NSLP), with a middle group having greater 
than 5 percent but no greater than 50 percent of students in 
NSLP. This variable also includes a category for students 
whose schools did not report their participation in NSLP. 

For purposes of this analysis, students in high-poverty 
schools were considered to be “disadvantaged,” while 
students in low-poverty schools were considered to be 
“highly advantaged.” The middle group was considered to 
be of mixed advantage. The variable is defined further in 
appendixes B and C of the full report. 

It should be remembered that there may be a fairly high 
correlation among some of these population variables. The 
report does not attempt to isolate the unique contribution 
of each factor to participation in vocational/technical 
education. Instead, the report describes bivariate relation- 
ships according to NCES standards for this type of analysis. 
See appendix C of the full report for additional information 
on the technical methodology used. 

Vocational/Technical Coursetaking in 1998 

Overall patterns among 1998 graduates 

Although most 1998 public high school graduates took 
at least some vocational/technical and occupational 
coursework, graduates who were members of disadvantaged 
groups generally took more vocational/technical and 
occupational coursework and were more likely to concen- 
trate in occupational education than their counterparts who 
were members of more advantaged groups. These differ- 
ences were apparent with regard to disability status in 
grade 12, GPA, academic coursework completed, grade 9 
mathematics, and school poverty. One exception was 
that students who had limited English proficiency in grade 
f 2 generally took less vocational/technical and occupational 
coursework and were less likely to concentrate in occupa- 
tional education than their English-proficient peers. 

In addition, male graduates took more vocational/technical 
and occupational coursework than female graduates, and 
students in rural schools took more such coursework than 
students in either urban or suburban schools. In contrast, 
Asians/Pacific Islanders generally took less vocational/ 
technical and occupational coursework than graduates in 
other racial/ethnic groups, particularly Black and White 
graduates. 



EDUCATION STATISTICS QUARTERLY — VOLUME 5, ISSUE 3, 2003 



35 




